While we're at it, I wonder if we should set xfail_strict = true for pytest?

It can be done through [tool.pytest.ini_options] -- see https://docs.pytest.org/en/6.2.x/reference.html#confval-xfail_strict.

Let me try if it fails any other tests.

Seems to work with not many more changes. Two tests need special attention: test_respect_order_in_requirements_file is xpass but I don't even understand why it's marked as xfail in the first place; test_no_clean_option_blocks_cleaning_after_wheel will need to be handled carefully in conjunction to #10485.

I'd prefer we merge this, and then deal with the fallout in that one. It already needs updating for a bunch of CI stuff (docs build) anyway.